OOC there are lots of boolean flags that are still bare ints. Are they all protected against races or?

probably not technically... I doubt the data race potential honestly exposes itself much in this area in practice as filling these in happens so much sooner than most anything that could be consuming them that all memory writes long since land. Py_IsInitialized() being a reader intended for concurrent access feels like an uncommon case. But I didn't try to analyze the others.

asking Opus 4.6 to analyze their use for races (my summary): _initialized, preinitializing, and preinitialized are all fine as used. They're serial during single threaded early internal init phases. Py_FatalError could wind up reading them for informational purposes but accuracy isn't important in that scenario.

[which makes sense to me]

🤖 New build scheduled with the buildbot fleet by @gpshead for commit abae231 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F146303%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

Thanks for doing this for PyO3's sake. Does thread-safe here mean that PyO3 could in principle freely call Py_Initialize from multiple threads and they will all block until whichever thread entered the actual initialization pipeline fully completes? I see atomic load/stores but no mutex hence the question.

(I appreciate what PyO3 is doing is slightly weird, because there's no way to reliably know which one of those threads becomes the "main thread". It's good enough for Rust test suites, at least!)

Sorry, I see I misread the title - this is about Py_IsInitialized being thread-safe; this won't make any difference to calling Py_Initialize and PyO3 will presumably continue to need to have a strategy to synchronize that call.

Does thread-safe here mean that PyO3 could in principle freely call this from multiple threads and they will all block until whichever thread entered the actual initialization pipeline fully completes?

No, CPython does not have a block until initialization completes API. That might make sense as a feature request on its own but I'd wonder what the practical motivation for it was and why we'd want people to do it.

I think it's fine to leave that synchronization downstream in PyO3 rather than complicate the C API. I think in most embedding contexts you want to be deliberate about interpreter lifecycle; what PyO3 does with the concurrent initialization is a quirk of Rust test suites launching multiple threads with no particular need for any proper finalization.

Agreed. We currently have a variety of C API surfaces (like Initialize and IsInterpreter) that are easier when embedding processes take a wide "hey i'm going to have a Python interpreter, let me go ahead and set that up once up front instead of having everything that wants to use it try to do that at time of first use" approach. That is not really practical for all situations. Your Rust example of using one from some test cases is a good example.

There are others, but they might expand into a larger C API and one interpreter vs multiple interpreters vs subinterpreters question. Imagine two independent libraries linked into a process unaware of each-other both wanting to use an embedded Python interpreter for internal use. As independent code, they have no good way to coordinate in a multithreaded context where they might both be trying at once. Which still ignores the question of if they need isolated interpreters or not. All of that could turn into a discuss.python.org thread...